Patched DataFusion version 45.0.0#54
Closed
wiedld wants to merge 14 commits intobase-df-upgrade-ver45from
Closed
Conversation
github-actions
bot
added
documentation
wiedld
force-pushed
the
iox-13280/upgrade-df-ver45
branch
from
010b01c to
6762c73
Compare
github-actions
bot
added
sql
logical-expr
functions
proto
catalog
development-process
labels
github-actions
bot
removed
sql
logical-expr
functions
proto
catalog
development-process
labels
wiedld
force-pushed
the
iox-13280/upgrade-df-ver45
branch
from
1353531 to
6405b35
Compare
alamb
changed the title
45.0.0
alamb
force-pushed
the
iox-13280/upgrade-df-ver45
branch
from
b1c4a07 to
c915be1
Compare
Author
|
This auto-closed. |
…rceDistribution) which later causes an error during EnforceSort (without our patch). The next DataFusion version 46 upgrade does the proper fix, which is to not insert the coalesce in the first place. test: recreating the iox plan: * demonstrate the insertion of coalesce after the use of column estimates, and the removal of the test scenario's forcing of rr repartitioning test: reproducer of SanityCheck failure after EnforceSorting removes the coalesce added in the EnforceDistribution fix: special case to not remove the needed coalesce
wiedld
force-pushed
the
iox-13280/upgrade-df-ver45
branch
from
b7d3c03 to
4f816b8
Compare
* Bump sccache version to latest to fix gh cache issue. * version blocked, trying with a hash * disable sccache.
* Bump MSRV to 1.82, toolchain to 1.85 * Fix some clippy warnings * Fix more clippy warnings
Author
|
The security audit CI will fail due to: But the pyo3 upgraded cannot be done until the arrow upgrade (Apr 4 commit: b717723) |
Author
|
I'm not going to debug the last few CI items, since they do not impact us. |
This was referenced Jul 14, 2025
|
Thank you for your contribution. Unfortunately, this pull request is stale because it has been open 60 days with no activity. Please remove the stale label or comment or this will be closed in 7 days. |
Author
|
We are past this version. Closing. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow on to
This brings us up to the version45 release, this Feb 3rd commit here and this apache branch.
Patches due to tech debt (all slated as lower priority, and are lingering):
chore: default=true for skip_physical_aggregate_schema_check, and add warn loggingPatches due to upstream bugs:
chore: skip order calculation / exponential planningUNIONandORDER BYqueries apache/datafusion#13748Patches because we haven't caught up yet:
fix: temporary fix to handle incorrect coalesce (inserted during EnforceDistribution) which later causes an error during EnforceSort (without our patch). The next DataFusion version 46 upgrade does the proper fix, which is to not insert the coalesce in the first place.fix(ci): build error with wasmSpecify rust toolchain explicitly, document how to change it (#14655)Bump MSRV to 1.82, toolchain to 1.85 (#14811)chore: get wasm to build in CI.Disable sccache action to fix gh cache issue (https://github.com/apache/datafusion/pull/15536)test: update sqllogictest errors.chore: fix clippy::large_enum_variant for DataFusionErrorPatches due to iox needs (unclear if upstream action TBD):
fix: handle when the left side of the union has no fields (e.g. an empty projection)count(*)where we aggregate to count rows after the projection.